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ABSTRACT 

Extensive job sample tests and multiple choice job 
knowledge tests were administered to approximately 370 men in each of 
four Army jobs: Armor Crewman, Repairman, Supply Specialist and Cook, 
Representative tasks in each job were analyzed and skill requirements 
were identified. This analysis and the correlations between job 
sample scores and job knowledge scores supported the proposition that 
knowledge tests are valid for measuring proficiency in jobs where 
skill components are minimal and where knowledge tests are carefully 
constructed to measure only information which is directly relevant 
for performing that job. (Author) 
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Abstract 

Extensive job sample tests and multiple choice Job knowledge testa 
were administered to* approximately 370 men in each of four Army jobs: 
Armor Crewman^ Repairman, Supply Specialist and Cook. Representative 
tasks in each job were analyzed and skill requirements were identified* 
This analysis and the correlations between job sample scores and job 
knowledge scores supported the proposition that knowledge tests are 
valid for measuring proficiency in Jobs where skill components are mini- 
mal and where knowledge tests are carefully constructed to measure only 
information which is directly relevant for performing that job* 
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THE INTKRCllANGEABILITY OF JOB SAMPLE 
TESTS AND JOB KNOWLEDGE TESTS IN FOUR ARMY JOBS 

Robert Vineberg and Elaine N. Taylor 
Human Resources Research Organization, Division No. 3^ 
When proficiency te»sting is undertaken to estimate the effectiveness 
of Job performance, assessment of both knowledge and skill is often 
required. If job sample tests are used in such assessment, they provide 
for the measurement ot both knowledge and skill since they are based upon 
samples of the actual performance required in a job. If job knowledge 
tests are used, only knowledge can be assessed since such tests are based 
upon an analysis, extraction, and sampling of the information required in 
job performance* They cannot be used to measure skill. Nevertheless 
practical constraints can be expected to promote the use of paper and 
pencil knowledge tests alone since they are simple and economical to 
administer. It is desirable therefore to determine where knowledge tests 
can reasonably be substituted for job sample tests* 

HumRRO research done under Work Unit UTILITY, which compared the job 
proficiency of men at different ability levels and with different amounts 
of job experience, provided an opportunity to e^camine the relationship 
between job sample test scores and job knowledge test scores in four Army 
jobs. These jobs> covering a fairly broad range of job types and task 
complexity, were Armor Crewman, Vehicle Repairman, Supply Specialist, and 
Cook. The Armor Crewman job consists principally of sequences of inter- 
action between a man and a weapon system with most tasks being largely 
procedural. The Repairman is a maintenance job Involving adjustment, 
trouble-shooting and replacement and repair tasks* The Supply Specialist 
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Job is clerical » calling primarily for coordination and recording of 
information* The Cook Job calls for preparation of food in accordance 
with a master menu and detailed recipes. Recognition of standards and • 
precision in meeting them are primary characteristics of this Job. 
* Representative tasks ii\ each of these Jobs were analyzed and the 

extent to which skilled behavior is required in the performance of 
individual task ateptf wd.s identified. This analysis and the empirical 
relationships between Job sample test scores and Job knowledge test 
scores supported the proposition that Job knowledge tests ar^"*- valid 
for measuring proficiency in Job^^whcx^;-^!) skill components are 
minimal 9 and 2) Job knowledge tests are carefully constructed to 
measure only that infoLmation which is directly relevant for performing 
that Job. 

Method 

Job sample tests and conventional znultiple choice knowledge tests 
were administered to a total of approximately 370 men in each of the 
four Jobs. In the construction of Job knowledge tests particular 
attention was given to include only information which was clearly 
relevant to Job performance* Each job sample test was individually 
administered and took from three and one-half to five hours to complete* 
Each Job sample test was comprised of a number of subtests. To make 
testing realistic and to simulate Job conditions^ each subtest consisted 
of the performance of a single entire task with a natural beginning and 
ending. Tasks were composed of a series of steps that would ordinarily 
be performed as part of a single operation in the performance of a job* 

Subtest tasks or problems were introduced to subjects Just as 
they might ordinarily encounter them* For example* on one of the 
Repairman subtests* a tank had been prepared so that the oil seals 
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leaked and the subject was told that oil was being thrown out of the 
tan.Js grill doors. The Repairman was to locate the source of the oil 
:ieak and repair the malfunction. In this problem there were 19 necessary 
steps. As an example, the first nine steps were: 1) w^S^ removes a 
fan vane by removing four screws securing vane to housing, 2) removes 
cotter pin, slotted hex nut, and flat washer, securing fan assembly to 
fan drive shaft, 3) Yemoves fan assembly from shaft, 4) reports fan 
tower seals leaking, 5) cuts and removes locking wire from bolts in oil 
seal housing, 6) removes six capscreys securing housing, 7) screws two 
5/16-18 X A bolts in threaded holes in oil seal housing, 8) turns both 
bolts in evenly until oil seal housing separates from fan drive housing 
cover, and 9) removes bolts from oil, seal housing. 

The Armor Crewman's Job performance was assessed on 20 subtests 
with a total of 354 steps to be performed and scored, the Repairman on 
13 subtests with a total of 175 steps, the Supply Specialist on 8 sub- 
tests with 156 steps, and the Cook on 5 subtests with 158 steps. 

As mentioned earlier, the behavior required in performing each step 
in each of the subtests was analyzed and categorized according to whether 
it was judged to be mediated by knowledge alone, a combination of cogni- 
tive skill and knowledge, or some combination of perceptual and motor 
skill and knowledge. 

To carry out the categorization of each step job knowledge was 
considered to be information about material used on the job and about 
acts, procedures, and principles needed to mediate job performance. 
If it was judged that the steps could be performed on the basis of 
information alone, it was categorized as requiring knowledge only. 

Job skill refers to a rather heterogeneous grouping of abilities. 
ERIC These abilities are inferred on the basis of behavior that is not 
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completely mediated by knowledge or verbalization of a to^be-per formed 
act. The acquisition of skill, being able to do something rather than 
knowing what to do, unlike the acquisition of knowledge, requires 
practice or rehearsal* Indeed, the need for practice in its develop- 
ment can be considered a major definitional characteristic of a skill. 
This characteristic is particularly Important for It provides a con- 
venient basis for deciding whether or not skill is being manifested In 
any given behavior* 

A related and common, thought probably not essential, characteristic 
of skill is that skilled behavior is typically not completely specifiable 
by verbal means even though the ends of such behavior are. For example, 
a person cannot be told by verbal instruction how to recognize subtle 
shadings in pitch in auditory tones or in a Gpoken language or how to 
swing a baseball bat. Tliese behaviors can be demonstrated but cannot 
be completely reduced to verbal description largely because the mecha*- 
nisms underlying them are internal and non-verbal (auditory processes 
in the case of discriminating tones, kinesthetic processes in swinging 
a bat) and language generally does not provide commonly agreed upon 
verbal labels for describing such processes. 

Four main classes of job skill can be identified: perceptual 
skills, motor skills, cognitive skills,, and social skills* While for 
some purposes it may be useful to distinguish additional varieties of 
skill, for example, artistic skills, these four classes seem to cover 
the critical varieties of skill in most jobs. If a step was judged 
to require some form of practice, it was categorized according to one 
of these three classes of i<*cill« None of the steps in any of the* 
performance tests required social skill* 
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Rcaults 

Analysis of the behavior required in aach task step revealed that 
the skill requirements in these jobs were minimal. Table 1 shows the * 
number of steps in each job sample test Judged to be mediated by know- 
ledge alone and those Judged to acquire cognitive skill or some combina-* 
tion of perceptual and motor skill. llfc^thf 



Insert Table 1 abput here 

Skill requirements in the Supply Specialist job are virtually non- 
existent. 

This analysis suggests that the correlation between job sample test 
scores and job knowledge test score:? in each of these jobs would be high 
and that in all likelihood the correlation would be highest in the 
Supply Specialist job. 

The (zero order) correlations between the job sample test scores and 
the job knowledge test scores are given in Table 2. 



Insert Table 2 about here 



Reliabilities for the job knowledge tests were: Armor Crewman - .81» 
Repairman - .76, Supply Specialist - .92, and Cook - .84. Only estimates 
of the lower limits of job sample test reliabilities could be obtained. 
The correlations given in Table 2 are substantial^ if the job knowledge 
reliabilities are taken aa the r«axiiaum possible value of test validity. 
These correlations » then» tend to support our proposition that job know- 
ledge testa can be appropriately substituted for job sample teats vhen a 
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Job contains little or no skill components and when only knowledge 
required on the job is included in the knowledge test. During the 
analysis of the job sample test step requirements > the skill components 
in the four Jobs were Judged to be minimal* Even in the Repairman and 
Cook jobs, where manual operations and the use of tools are common, 
skill requirements appeared to be small. It may be assumed that most 
persons are able to. perform in either of these Jobs almost completely 
on the basis of knowledge both of the required tasks and of the specific 
operations entailed. As anticipated the highest correlation occurred 
between Job sample and Job knowledge criteria in the Supply Specialist 
job. This relationship is of particular significance for this analysis. 
A clerk^s Job| typified by the Supply Specialist, represents one of the 
purest examples of a Job where knowledge rather than skill is sufficient 
to support performance. 

Given the high costs associated with Job sample criterion testing, 
not only will job knowledge tests be used, but they should be used 
where skill requirements in a job are minimal* 
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Footnote 

1. The research reported in this paper was performed at HumRRO 
Division No. 3, Monterey, California, under Department of the Army 
contract with the Human Resources Research Organization; the contents 
of this paper do not necessarily reflect official opinions of policies 
of the Department of rhe Army. Reproduction in whole or in part is 
permitted for any purpose of the Department of the Aray# 



Table 1 

Frequency of Steps in Job Sample Tests Categorized 
According to Knowledge and Skill Requirements 

Armor Repair Supply Cook 
Knowledge alone 338 165 153 U5 

Cognitive skill and knowledge 0 A 3 11 

Perceptual^otor skill and 16 6 0 2 

knowledge 

Total No. of Steps 3^4 175 156 158 



Table 2 

Zero Order Correlations Between Job 
S<iinple and Job Knowledge Teat Scores 

Armor (N-368) .68 

Repair (N-360) ,59 

Supply (N-380) .72 

Cook (N-366) .58 



^All correlations «kre significantly 
different front zero (p<.05). 



